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Abstract 

We consider an ensemble which interpolates the Laguerre orthogonal ensemble and the Laguerre 
symplectic ensemble. This interpolating ensemble was introduced earlier by the author and Rains 
in connection with a last passage percolation model with a symmetry condition. In this paper, we 
obtain a Paineleve V expression for the distribution of the rightmost particle of the interpolating 
ensemble. Special cases of this result yield the Painleve V expressions for the largest eigenvalues of 
Laguerre orthogonal ensemble and Laguerre symplectic ensemble of finite size. 



1 Introduction 

Given a sequence £ = (£i, • • • , £jv), the Vandermonde product of £ is denoted by 

l<i<j<N 

For any real constant A > —1/2, we consider the probability density function defined by 

N 

p(&,.-. ,H N ;A) = Z N ^ A A N (Ol[e-^e A (- i y^ (1.2) 



on the ordered set {0 < £n < • • ■ < where 

JV-l 

2 



1 N ^ 

^ = (^+^11^ (1-3) 
is the normalization constant. The main purpose of this paper is to express the distribution of the 



rightmost 'particle' £i in terms of a solution of the Painleve V equation (see Theorem 1.5 below) 



There are two reasons that we are interested in the above density. The first is that (1.2) interpolates 



the Laguerre orthogonal ensemble and the Laguerre symplectic ensemble in the random matrix theory, 
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as A varies from to +00. Letting A — > or A — > 00 in the Painleve expression, we verify in particular 
that the distribution function for the rightmost particle £1 satisfies the Painleve V equation for the 
Laguerre orthogonal and symplectic ensembles, respectively. These are new results in the literature. 
The second is that the above density function represents the probability distribution of a certain last 
passage percolation model with a symmetry condition. Indeed, the density ( |l.2| ) was introduced in 
(see Remark 7.6.1) as a formula for the distribution of the last passage time in this percolation model. 
We now discuss these two aspects of the above density function. 

Interpolating ensemble 

We first discuss the connection to the Laguerre ensembles. Let w(x) be a weight function on R or on a 
subset of K which decays sufficiently fast as x — ► ±00. Consider the density function 

1 N 

-A^n^) (1.4) 

3=1 

on the set £jv < ■ ■ ■ < £1, where /3 > is fixed, and Z is the normalization constant. The ensemble with 
the special choice of weight function 

w(x) = x a e~i x l x > (1.5) 

is called the Laguerre orthogonal ensemble (LOE), Laguerre unitary ensemble (LUE), and Laguerre 
symplectic ensemble (LSE) for j3 = 1,2 and 4, respectively (see e.g. p2[). The Laguerre ensembles 
are of basic interest in the multivariate analysis of statistics (see, e.g. @]). Especially, the LOE with 
a = M — 1 — N represents the probability of the principal components (i.e. the singular values) of an 
M x N random matrix X whose entries are independent (real) Gaussian random variables of mean 0, 
variance 1. 

Introduce the density function, for real A, given by 

N 

p(&,..- ,Zn;A;w;I3) := Z^ A , W .^A N (O JJ "(fc)^- 1 ^ (1.6) 

3=1 

on the set M.^ rd := {£jy < £n-i < • • • < This density function generalizes and also interpolates the 
ensembles ( ]1 .4j ) for different /3's in the following sense. 

Proposition 1.1. When A = 0, 

N 

p(fc,--- ^ N ;0;w;/3) = Z^ . w A N (0^l[w^ j ) (1.7) 

3=1 

with some constant Z(N,0;w). Let w(x) = e~ v ( x ' be a positive, C 1 -function supported on a subset of 
M. such that V{x) > cq\x\ for some cq > as x — > ±00, and ||V||j,«. < Cq. Assuming that N is even, 
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for a bounded uniformly continuous function /(£i, • • • ,£n)> we have 



lim / f(£ lt --- ,£ N )p(£i,--- ,£ N ;A;w;P)dti---d€N 



r N/2 

3=1 



(1.8) 



where g(Ci,C2, ■ • • ,Cn/2) ■= / '(Ci, Ci, C2, C2, ■ • ■ , Civ/2, Cw/2) 



N ,oo;w;fl 



We also have for any t G M 7 



lim 

A— i-+oo /raJV 



d n{£i<t} 



JV/2 

A W/2 (o 4/3 ri w (o) 2 do 

3=1 



(1.9) 



A' 



AT/2 

A(o 4/3 ri w (o) 2d o 

3=1 



(1.10) 



The case ^4 = is trivial from the expression dl.q ). An explanation for the change [3 — > 4/3 in the 
case when ^4 — ► +00 is the following. We first note that the term involving A in (|Lq) is 



-A(fi-£ 2 +£ 3 -f4+-) 



(1.11) 



Since £j's are ordered, £1 > £2 > • • • > S.N (N even), the term £1 — £ 2 + £3 — £4 + • • • is always non- 
negative. Thus as A — > +00, we have a non-trivial limit for (|l.6| ) only when £1 = £2, £3 = (,4, ■ • • (the 
term Z^f A . w .g grows polynomially in ^4 ; see Lemma 2.1 below). If we set (1 :— £1 = £2, C2 := £3 = £4, 
• • • , simple algebra shows that the Vandermonde term Ajv(£) in (1.6) becomes Atv/2(C) 4 if one drops 



the terms (^2fc-i — &k)i 1 < k < N/2, which vanish. (See also the Remark in Section g below for 
changes (3 — > fc 2 /3, fc e N). The full proof of this Proposition is given in Section ^J. 

In view of the Coulomb gas interpretation of random matrix theory (see e.g. pi|), the term involving 
A in the density ( |1.6| ) represents the pairwise-attraction of particles, in addition to the log repulsion 
given by Ajv(£) and the "external field" w. As A becomes large, the pairwise-attraction becomes 
stronger, and for each j — 1, • ■ • , N/2, the pair of particles £y-ii 6y gets closer, and eventually sticks 
together to form one particle Q. 

If we take w(x) = e~^ x on x > and (3 = 1 in the above Proposition, ( |l.2j ) becomes the density 
for LOE (1.5) with a = when A = 0, while the limit of (1.2) as A — > +00 is, with the modification 
e _ 3^ by ', the density for LSE (1.5) with a = 0. Thus by taking A = and A = 00, the Painleve 



V expression (Theorem L3 below) for the largest particle of the interpolating ensemble ( |l.2j ) yields the 
Painleve V expressions for the largest eigenvalue of LOE and LSE (see Corollary 1.4 below) with a = 
for any finite N € N. 

In the random matrix context, there have been many works that express the various distributions 
of matrix ensembles in terms of differential equations (see e.g. |3(| and the references therein). Most 
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of results are for the unitary ensemble {(3 = 2), and there are relatively few results for (3 = 1 and (3 = 4 
( [p8[ [lij [y| [l7|, |l(|). For the Laguerre ensembles, the probability distributions for (3 = 2 are found 
to be expressible in terms of Painleve V equation ((2(|) for any finite N, and in terms of the Painleve 
II equation (|p7|) in the limiting case N — > oo. On the other hand, for /3 = 1, Johnstone Q^o)) recently 
analyzed the limiting case ./V — > oo, and obtained a Painleve II expression. The Painleve V expressions 
for LOE/LSE for finite N in this paper are new. While this paper was being written, Forrester and 
Witte obtained different formulas for the largest eigenvalues of LOE and LSE (fill). Their formulas 
involve Painleve III' systems instead of Painleve V. The relationship between these two formulas is an 
intriguing question and remains unclear. 

Last passage percolation with a symmetry condition and the totally asym- 
metric exclusion process with symmetry condition 



As mentioned earlier, the density function (|l.2[ ) also arises in connection with a last passage percolation 
model. For r > 0, let e(r) denote the exponentially distributed random variable with mean r : the 
density function of e(r) is r~ 1 e~ x ' r for x > and for x < 0. By e(0) we understand the random 
variable identically equal to 0. Fix p > 0. To each site € Z+, we attach a random variable u(i,j) 
taken as follows : 

u(i,i)~e(l), i<j, (1.12) 
u(i,i)~ e (p), (1.13) 
u{j,i) = u(j,i), i<j. (1.14) 

Except for the symmetry condition u(i,j) = u(j,i), the random variables are independent. Note that 
the condition (1.14) implies the symmetry of the configuration of random variables in about the line 



y = x. An up/right path ir is a collection of sites {(ik,jk)}k=i satisfying (ifc+i, jk+i) - (hi 3k) = (1)°) 
or (0, 1). Let II(iV) be the set of up/right path n from (1, 1) to (N, N). Define the random variable 

H (N;p) =max{ ^ u(i, j) : tt G U(N)}. (1.15) 

If one interprets u(i,j) as the (random) passage time to pass through the site (i,j), H a (N; p) is the last 
passage time to go from (1, 1) to (N, N) along a directed (up/right) path. The relation of H a (N; p) to 
the above interpolating density function is the following. 

Proposition 1.2. (Remark 7.6.1 We have 

r i i N 

¥(H*(N;p)<x)= ,,(£,... -jTTdg,.. (1.16) 

J0<£N<-<tl<X P L 



This Proposition is stated without full proof in Remark 7.6.1 a full proof if given in Section [| 
below. 
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Closely related is a one-dimensional interacting particle system, the totally asymmetric simple exclu- 



sion process (TASEP) ( [|2 1| . 19 1). TASEP is a continuous-time stochastic process on the integer lattice 
Z. At any time, each site is either occupied by a particle or empty. If a particle is at a site j and its 
right-hand-site j + 1 is empty, the particle jumps to its right-hand-site after a random waiting time 
given by an exponential random variable of mean 1. Thus the particles move only to the right. The 
waiting time for the jumps is independent and identically distributed at each site and each (continu- 
ous) time. These rules describe the usual totally asymmetric simple exclusion process (see e.g. pj| , 
[ 53 , pl[). In p9| , Johansson showed that the TASEP with special initial data (all negative sites are 
occupied and all non-negative sites are empty) can be mapped to the last passage percolation above 



without the symmetry condition (1.14). This mapping was further generalized in p4| for TASEP with 
random Bernoulli initial data. 



The above last passage percolation model with the symmetry condition (1.14) is also related to a 
TASEP, but now the process is defined only on the non-negative integer lattice, No = NU {0}. For the 
sites j > 0, the jump rules remain the same as before, but we assume that there is a creation process at 
the origin j = : when the site j = is empty, a particle is created after a random exponential waiting 
time of mean p. For initial data, we assume that all the sites are empty. Then one can show that (p3j) 
the number of particles N(t) that have been created at the origin up to time t satisfies 

P(N(t) < N) — P(H a (N; p) > t). (1.17) 

In the last passage percolation model above, one might also be interested in the last passage time 
from (1,1) to (M,N) for general M ^ N. In terms of TASEP, this is equivalent to the number of 



particles that have jumped across the site M — N. But there is yet no formula like Proposition 1.2 for 
general M ^ N. 

Results 



Now we state the main results. From Proposition 1.2, the following results imply the Painleve V 



expressions for the distribution of the rightmost 'particle' from the interpolating ensemble (1.2). 
Theorem 1.3. With the notation — and w = - p — 1 = 2A, we have for x > 0, p > 0, N G N, 

P(H®(N; p)<x) = i j [a N (x, p) - b N (z, p)] (E N (a;)) _1 + [a N (z, p) + b N (x, p)] E N (z) j F N (z) (1.18) 
for some functions 

~a(y,N)dy\, (1.19) 



F N (x) := cx p|y \ l 

E N (x):=exp|y -/%; A)dyJ (1.20) 

and &jq(x, p),bjq(x, p). The functions a(x), P(x),a-^(x, p),b-^(x, p) are analytic in x > 0,p > 0, and 
satisfy the following properties : 
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(i) (Painleve V) The functions a(x; N), f3(x; N) as functions in x satisfy 

a'(x;N) = ^([3(x;N)) 2 . (1.21) 
The function 9{x) = 0(x;N) := —^xa(x) solves the Painleve V equation 

{xO"f = {6- x6'){0 - x6' + 4((9') 2 + ANff). (1.22) 

(ii) (Asymptotics of a, (3 as x — * oo) Fix < e < 1/4. For each fixed N 6 N, as x — > +oo, 

(3{x) = 2(-fL« 1 (x)e^ 1 + 0{e-&-*') = ^^"^(l + 0(a: -i)) (i. 23 ) 



and 



«(*)=/ 2(4 ) _ 1 ( !; )) 2 e-^ + Q( e -l' 1 -»)= - e -(i + Qfo-i)), (i. 24 ) 



((7V-1)!) 2 

where L^_^(x) is the Laguerre polynomial of degree N — 1 with parameter 1 ; (see, e.g. W) 



(m,) ('Lax pair for Painleve V) The functions a,^{x,p) and h^(x,p) are real and smooth in x > and 
p > 0, and they satisfy the differential equations (note w = - — 1) 

and 

9 fM _ i /b N \ i f-QoQ' MVM.ijLrfM (127 ) 

dw\& N ) 2\0J l-w 2 \~( x py (xa)7 W/ 2 1 - ™ 2 V W ' 
fiy) (Asymptotics of a^,b^ as x — > ooj For any < e < 1/4, as X —> oo, we /iai>e 

Mx,p) = { H - 2 -^ (1.28) 



$(w;a;)(A 1 (-w,a;) +0(e-( 1 - 2e ) :E )), p > 



2 

2-e ' 



-A 1 ( W ,x)+0(e-( 1 - 2 ^$( W ;x)), < p < §, 

b N (a;,p) = < (1.29) 

-$(w;x)(l + 0(e-( 1 - 2£ ) a; ), p>^, 



where 



Ai(w,x,N) := -L /" $( s;a; )^ ; $( w;x):= 6-^(11^)^. (i.ao) 

27T2. 7| s _l| = ^|l_^| S-1U V l-W' 
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(v) (Asymptotics o/aN,bN as p — ► + ,2,ooJ FFe ftaue 



lim ajsrfx, p) = 1, lim bN(£,p) = 



and 



(1.31) 



a N (x,2) = E N (x) 2 , 



b N (a;,2) 



E N (a;) 5 



(1.32) 



^4Zso /or /^erf y > 0, 



lim a N (yp, p) = P(iV, y), lim b N (yp, p) = 0, 

p — >OC p — *oo 

where P(N,y) is the incomplete Gamma function (see, e.g. Jljj) 



P(N,y) = 



(N - 1)! J 



(1.33) 



(1.34) 



Remark. The existence of the solution 8 (hence a) to the Painleve V equation ( 1.22 ) with the asymptotic 
condition (1.24) is a part of the Theorem. But as yet we do not have uniqueness for 8 (or a) as a solution 
of ( 1.22 ) with asymptotic condition ( 1.24 ). Another missing piece of information is the asymptotics of 
a and j3 as x — > + . These issues will be studied in a later publication. 



From Proposition 1.1 , by using ( 1.32 ) and ( 1.31 ), Theorem |L3j implies the following results for LOE 
and LSE at the special values p = 2 (A = 0) and p — > + {A—* oo). 

Corollary 1.4. We have, with the notation := {£,n < 6v-i < • • • < / or arl 2/ 1 > 0, 



JV 



Ajv(0 J] e~^d^ = V(H a (N; 2) < x) = E N (x) F N (z), (1.35) 



and /or AT even, 



=lv3rT / w/2 (Aiv/2(C)) 4 II e_Cid = P(tf (7V;O) < *) - A (En(^)) 1 +E N ( 2 ;) F N (a;). 

(1.36) 

Remark. Once the uniqueness of (or a) is proven (see Remark above), and also the uniqueness of f3 
is established, the above Corollary provides a tool for numerical computations of the largest eigenvalue 
distribution of LOE and LSE for the special case w(x) — e~i x (see (1.5)) for any finite N. 

For LUE, the largest eigenvalue distribution was obtained by Tracy and Widom (pq|) : 



N 



(A N (0) 2 Y[e~^d^ = (FMx)f 
rf «)n{6<4 J= i 



(1.37) 



Note the special structure of the formulas ( 1.35| ), (1.36) and (1.37), from which an interesting inter- 
relationship of the largest eigenvalues of LOE, LUE and LSE can be derived. We refer the reader to 
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for a full discussion on inter-relationship between orthogonal, symplectic and unitary ensembles. 



We also remark that Corollary 1A applies only for the case a = of the Laguerre weight (1.5). We do 
not have results for LOE and LSE of other values of a. On the other hand, LUE with different values 
of a 7^ was analyzed in |26|| . 

If we take the limit p — > oo (A J, —1/2 ; note that (1.2) is defined for A > —1/2), we have 
P(iJ (N; p) < x) — > for fixed x > 0. To obtain a non-trivial limit from ( 1 .32 ) we set x = yp 
and let p — > oo while y > is fixed. Note that from (1.23) and (1.24), we have Fj\j(x) = 1 — 



1 ;X 2N - 2 e- 



(1 + 0(x~ 1 )) and E N (x) = 1 



2((iV-l)!) 

we find that En(x) Fn(x) — > 1 and En(x) -1 Fn(x) 
Corollary 1.5. For any fixed y > 0, 



(N-iy. 
— ► 1 as x 



N x e 2 X (1 + 0(x x )) asu oo, and hence 



lim ¥{H a {N;p) < yp) = P(N,y), 



(1.38) 



where P(N,y) is the incomplete Gamma function (1.34) 



This result is consistent with the intuition that when p is large, the longest up/right path in the 
percolation model is basically the diagonal line through the points (i, i), i = 1, • • • , N. As p — > oo, we 
expect that the random variable H®(N; p) is close to Spf(p) := u(l, 1) + u(2, 2) + • • ■ + u(N, N), the 
sum of N i.i.d. exponential random variables of mean p. A direct calculation shows that ¥(Sn(p) < 
a) = P(N, a/p). Thus Corollary O shows that indeed H a (N; p) - S N {p) when p 



oo. 



This paper is organized as follows. The proof of Proposition 1.1 is given in Section |[ In Section ||, 
we consider a different percolation model which has a geometric random variable at each site instead of 
an exponential random variable. Since a geometric random variable converges to an exponential random 
variable in an appropriate limit, the related percolation model with geometric random variables also 



converges to the percolation model with exponential random variables (see Lemma 3.1 below). In Section 
H and Section |], we present two different formulas for the geometric percolation model. A multi-sum 
formula is given in Section |], and in an appropriate limit, this multi-sum formula becomes the multi- 
integral formula in Proposition 1.2 for the exponential percolation model. In Section [|, we express the 
distribution for the geometric percolation model in terms of orthogonal polynomials (see Lemma 4.2 
below). The appropriate limit of these orthogonal polynomials yields the Painleve expressions, Theorem 
1.3. The asymptotics of orthogonal polynomials are obtained in Section a by applying a steepest-descent 



method to the associated Riemann-Hilbert problem, and the proof of Theorem [L3j is given at the end 
of Section |^. Some properties of the Riemann-Hilbert problem for the Painleve V equation, which will 
be used in Section ^ are summarized in Section 0. Section [?] presents some results for the percolation 
model as A — > oo. Finally, in Section || we show that the computation of Tracy and Widom (|2!j) which 
expresses the correlation functions for orthogonal and symplectic ensembles in terms of determinants 



can also be applied to the density (1.6) with f3 = 1 which includes fll.2| ) as a special case. 



Acknowledgments. We would like to thank Percy Deift, Eric Rains, Peter Forrester and Xin Zhou 
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2 Proof of Proposition 



The identity (1.7) is trivial. We prove (1.8) in this section. We basically prove that when A — > oo, the 
function p(£i,- ■ ■ , £at;A;7«) on M.^ rd will concentrate on the subset satisfying £2.7-1 = £2.7 f° r eacn 3- 
We present the proof only for the case when (3=1. But it would be clear that the proof for general 
[3 > will be the same. In the below, we omit any dependence on f3. 

We assume that N is even and A>\. Substitute £2.7-1 = £2.7 + x j, j = 1 } ■ • • , N/2, and set Q = £2^. 
We use £1, • • • , £/v/2 and x\, ■ ■ ■ , as new variables. Then the region of integration {£/v < £jv— 1 < 
• • ■ < £1} becomes the set {Q N/2 < ■ ■ ■ < £1} U {0 < x t } U {0 < Xj < - Q,j = 2, • • • , N/2}. We will 
denote by M+ /2 (£) the set {0 < Xi} U {0 < xj < £j-i - (jj = 2, ■ ■ ■ , N/2}. In the below, we use the 
notation c?£ = Yifli dQ an d dx = flfci dxj, an< ^ a l so w (0 — Yifli w (Cj)- Then the integral on the 



left hand side of (1.8) without the limit A — > +00 becomes 

AT/2 



(*) :=^,A;» /"(CK / fp(x,Ol[e-^w(x j + C j )dx j , (2.1) 



where 



N/2 

p( x i = n Xi n (^-^+^-0) 

i=l l<i<j<N/2 ^2 2) 

x n (*i+Ci-o) n (Ci-o-^) n 

l<i<j<N/2 l<i<j<N/2 l<i<j<N/2 

is a positive polynomial, and also each factor is non-negative. 

Let < e < 1 be any fixed number. Let B^ 2 := {0 < Xj < e, j = 1, • • • ,N/2}. We divide the 
integral with respect to x into two regions : (1) X\ := K^ /2 (£) n sf /2 and (2) := M+ /2 (£) \ X x e . 

We first show that Zj^ A . w has a polynomial growth as A — > 00. 

Lemma 2.1. We have 

lim A n Zn a- w = Z}f,oo;w (2-3) 

A— >oo 

where Z/v,oo;tu *s defined in ( |1.9| ). 

Proof. For a; € X|, since e _A:Cj < 1 and there is at least one Xj satisfying Xj > e, we have 



/ w(()d( f P(x, £) n e ^ W (x, + Q)d Xj 

■ jR ord ^ X 2 j=l 

r r N/2 



(2.4) 
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For x £ Xf, since ||V|U°°(K) < C , we have w(C)e~ eCo < w{x J + Q) < w{Q)e eCo for all 1 < j 
Thus 



J N,A\w 



< e5 ( 



N/2 N/2 

hNc ° / N/2 n w (ctf d (j / c) n z- Ax] d* 

J^ or d j = l ^ X ' - — I 



i 3=1 



and 



r N/2 

keNc ° / N/2 n w (Q) 2 dQ / p(x, o n e ~ Axjdx 

JM ar-d j = l JX ' - — 1 



AT/2 

n 

3=1 



In Q2.2D , for x e Xf and ( £ > we have 

AT/2 

p(x, o < n ri( e + c< - o) n> + c< - s ii u - co 11^ - co 

i= 1 i<j i<j i<j 

N/2 

< (A N/2 (0 4 + eQ(0)l[x k , 

k=l 



where A is the Vandermonde product (1.1), and Q(C) is a positive polynomial. Thus using 

xeT Ax dx = A~ 2 , 



(2.5) is less than or equal to 

N/2 

A — 1 



3 + e A£ ZNfi;w- 



But 



r N/2 

2 &N,2{Q A X{w{Q) 2 dQ = Z 



N,oo;w: 



and hence 



Zn,a-w < -jjy e 2eNC "(ZN,oo;iu + eCi) + e A€ Zn,0\c 



for some constant Ci > 0. This implies 



Similarly to (2.7), we have 



limsupA Z N ,A\W < Z Nt00;w . 

A~*oo 



N/2 

P (x,0 > (A N/2 (0* - eR(Q) I] x k 

k=l 
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for some positive polynomial R(Q. Hence from (2.6), 

N/2 N/2 



Z N , A;w > e-^ NC ° f N/2 (A W/2 (C) 4 - eR(C)) f[ w(Q) 2 dQ f f[ ,y u <l.r,. (2.14) 



Since X| = M+ /2 (C) n Sf /2 , we have 

N/2 N/2 



I ft ' J = IK 1 - ( A + l)e- A ^^~^), (2.15) 



where Co := +oo. We note that (2.15) is bounded below by A N (1 — (A + l)e Ae ) N , and above by 
A~ N , if we take A large enough. Hence by considering (|2.14| ) as sum of two integrals, one involving 
Aj^ 2 and the other involving R, we find that 

Z N , A;w > e -ieNC Z^ {1 _ {A+ 1)e . At)N _ ee -hNC O2_ (2 16) 

for some constant C2 > 0. Therefore we obtain 

<liminfA Zn,a-w (2-17) 

□ 

Proof of flTJ) 

Fix < e < 1. Then there is a i5 > such that for a; G , we have 

|/(Ci + x u Ci, C2 + x 2 , C2, • • • ) - ff(Ci, C2, •••)!<£• (2-18) 



We write (O) as 



(*) = (**) + (*1) + (*2) + (*3), (2.19) 



where 



r r N/2 

(**) = / N/2 WCK / N/2 P(«, C) I] + ( 3 )dx 3 , (2.20) 



and 



(*1) = Z^ A . W [ w(t)d( [ fp(x,0t[e- A ^w(x J+ Q)dx 3 , (2.21) 

(*2) = -Z N ] A . W gw(0d( p{x,Ol[e-^w(x j + C j )dx j , (2.22) 
^"/d Jxi j=1 

r r N/2 

(*3) = / u>(CK / (/ - <?M*> II e-^w( Xj + Q)dx 3 . (2.23) 
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As in (2.4), we have 



|(*l)|<||/||z~e- 



-AS Zn,0;i 



J N,A;w 



On the other hand, from (2.18) 



l(*2)| < II.9II 



N/2 



-AS Zn,0;w 



J N,A;w 



|(*3)| <eZ^ A . w [w(C)dC I P(x,0f[e- Ax > 

ord 1 J — ± 



w(xj + Q)dxj < e. 



(2.24) 



(2.25) 



Thus using Lemma 2.1, limsup^^,^ |(*1) + (*2) + (*3)| < e, but e > is arbitrarily small, hence we 
have 



lim |(*) - (**)| = 0. 

A^oo 

Now the only remaining thing is to show that as A — > oo. 



(**)- Z N ]^ t . l a 



N/2 



n (Ci-o) 4 nwo)) 2 ^i' 

ord l<i<j<N/2 j=l 



Fix < e' < 1. We write 



(**) = (* * 1) + (* * 2) + (* * 3), 



where 



N/2 



, N/2 

gw(()d( K^OTT' 

J*j 3 = 1 



e 3 w(xj + (j)dXj, 



N/2 



(**2) = Z N \. W \ gw(0cl{ [ i p(xX)f[e- A *t(w(x j +( j )-w(<; j ))do 

J </i Jx t 3=1 
(* * 3) = z N ) A . w / 5W (C) 2 rfC / , p(x, C) I] e-^^-. 



As in (2.4), we find 



|(* * 1)| < \\g\\ L ~Zrf A e~ M 'Z Nfi , w -> 



(2.26) 
(2.27) 

(2.28) 

(2.29) 
(2.30) 
(2.31) 

(2.32) 



as A — > oo, using Lemma 2.1. For the estimation of (**2), we use \w(xj + Q) — w(xj)\ < D(e')w(xj +Cj) 
for acj- € Xf', where D(e') := max(|l - e e ' Co |, |1 - e- £ ' c °|), and find that 

\(**2)\<D(e>)\\g\\ L ~Z N ^ w w(()dC / p(x, Q JJ e -^w(x s + Q)d Xj < D(e')\\g\\ L ~ (2.33) 



,;=i 



where the second inequality is obtained by replacing the region X\ by R+^ 2 (£) and noting the the total 



integral is 1 by (2.1). For (* * 3), we note that (2.7), (2.8) and Lemma 2.1 yield 

N/2 N/2 







< Z N ] A , W f , P(*, fl e-^'dxj < (A N/2 (Cr + e'Q(C))Z N ^ w / , fl 



c ^ dx j 



(2.34) 



< 



(Ajv/ 4 (C) 4 + e'Q{Q)Z-] A ^A- N < (A W/4 (C) 4 + e'Q(0)cZ^' 
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for some constant c > 1. But Atv/ 4 (£) 4 + e'Q(C) is integrable with respect to the measure w(() 2 d( on 
^ord' ano - nence we can use t ne Lebesgue dominated convergence theorem. But an argument similar 

N/2 
ord ' 



to (^7|)-( [2.17D shows that for each ( e 

N/2 



lim ^v 1 ^ / p(*, II e-^'dxi = Z^An/M) 4 , (2-35) 



and we find 

2V/2 



lim (* * 3) = Z- N )^ w f g J] (C 4 -0) 4 IlM0)) 2 rf0- (2.36) 



ord l<i<j<N/2 4=1 



Thus the estimates (|2.33D , (j2.36|) , together with the fact that D(e') -> as e -> 0, yield fl2.27|) , 

and we prove the Proposition |l.l| . 



Proof of dLTO) 



In the analysis above, the fact that / is uniformly continuous is used only for the estimation of (*3). 
Now when / = l^ N <-<^<t = lo< Kl <t-Ci,Ci<t an ^ 9 = ki<t' becomes 

(*3) =Z N \. W / w/2 ^(CR / P(x, C) I] e-^ W (^ + Q)dx 3 

jR oJd n {Ci<t} Jxln{o<xi<t-Cx} ,- =1 

(2.37) 

W/2 



' K' 4 J ^<t} J=i 



where now < 8 < 1 can be taken to be an arbitrary fixed constant. When t — £i > (5, X* n {0 < xi < 
t — d} = X[, and the above two integrals in x are the same, and hence we have 



|(*3)| < Z- N ) A f w(C)dC f p(x,C) J] e- A ^w(x J+ Q)dx v (2.38) 

ir' il ! n{t-J<(i<i} 7x 1 5 n{t-Ci<a;i<<5} j = i 

As in Lemma |2.1| , this can be estimated as 

l(*3)|<^ ; ^ e ^ c " / (A N/4 ((f + 5Q(0)w(0 2 dC (2.39) 

But Z7r A A N is bounded as A — > oo, and the integral vanishes when we take 5 — > 0, and hence we 



obtain (*3) — > as A — * oo. The rest of analysis is the same as for the proof of (l.S). 
Remark 

In addition to the change f3 — > 4/3, we can also obtain the transition f3 — > fc 2 /3 for each fc € N. Let 

:= e -^(^+^- 2 «3+S 4 +e5-2£ 6 +...) j (2 4Q) 
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and similarly we set A) with the term £j + £ 2 + ■ ■ • + Cfc-i — (k ~ l)6fe + £fc+i — • • ■ • Then when TV 
is a multiple of fc, 

iV 

lim Zv',:,:^ / /(OAjvCO^fefe^n^fe)^ 

, (2.41) 

- ^oo;^ / N/fc ^(OAiV/^C)^' 3 n^(0)) ferf 0- 

where //. is obtained from / by setting the first k variables equal, and the next k variables equal, and 
so on. There are other possible choices of rjk- For instance, 

n k (&A) =e -^(«i-^+^+i-^ fc +-) (2.42) 



would again yield (2.41). 



3 First formula for geometric percolation : multi-sum expres- 
sion 

For < q < 1, let g(q) denote the geometric random variable with parameter q : for k = 0, 1, 2, • • • , 
P(g(q) = k) = (1 — q)q k ■ We consider a last passage percolation model with geometric random variables 
analogous to the percolation model with exponential random variables considered in the Introduction. 
Namely, to each site £ the random variable X(i,j) is attached where 

X{i,j)~g(q), i<j, (3.1) 
X(i,i)~ g(ay/q), (3.2) 
X(j,i) = X(j,i), i<j- (3.3) 

Here q £ (0, 1) and a £ (0, 1/yfq) are fixed numbers. As before, the random variables are independent 
except the symmetry condition X(i,j) — X(j,i). As in the Introduction, II (TV) denotes the set of 
up/right paths from (1,1) to (N,N). We define 

G (/V;a) = max{ ^ X(i,j) : 7r £ II(iV)}. (3.4) 

Since a proper limit of the geometric random variable becomes the exponential random variable, we 
find that the random variable H a (N] p) is a limit of G a (N.a). 

Lemma 3.1. We have 

P(H a (N;p) < x) = lim P(G (/V;a) < xL), (3.5) 

L — *oo 

where we set 

^ = 1 -2T< a = 1 -(H)l' (3 - 6) 
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Proof. It is direct to check that with (3.6), 

e(l) = hm — — , e(p) = hm f — , (3.7) 

L — >oo Li L — >oo Li 

in distribution. Thus under this limit, the last passage percolation model with geometric random 
variables becomes the last passage percolation model with exponential random variables. It is also 
direct to check that G (N, a) /L -> H a (N, p) in distribution. □ 

Now the key thing is that there are two different formulas for P(G (7V, a) < n). Thus by taking the 
exponential limit L — » oo with (3.6), we would obtain two different formulas for F(H a (N, p) < x). It 
will turn out that one of the limiting formula is the multi-integral formula given in the right-hand-side 
of ( 1.16| ) which represents the probability distribution for the rightmost 'particle' in the interpolating 
ensemble, and the other is the Painleve V expression, Theorem 1.3. We present the first formula for 
P(G (A^, a) < n) in this section. The second formula will be considered in the subsequent sections. 

The following lemma is modeled on the paper fl9|| in which a similar result for the unsymmetrized case 
(at each site the geometric random variables X(i,j) are independent and identically distributed 

without the symmetry condition X(i,j) — X(j, i)). 

Lemma 3.2. Let N = N U {0}. We have 

N 

F(G m (N;a)<n)=Z 1 (N,a)- 1 £ J] (h t - hj) J] q h ^ 2 a -^ ih ' , (3.8) 

0<h N <---<h 1 <n+N-ll<i<j<N i=l 

where the normalization constant is 

JV-l 

Zx(N, a) = (l- a^q)~ N (l - q )-N(N-l)/2 a [N/2] q N(N-l)/2 -Q j{ (3 Q) 

3=0 

Remark. The result for the special case when a = 1 is stated in Remark 5.2 Jul. 

Proof, (cf. Section 2.1, |Q and Proof of Theorem 7.1, Q) Let A = (Ai, • • • , Xn) is a Young diagram : a 
sequence of integers, Ai > ■ • • > Xn > 0. The number d\(N) of semistandard Young tableaux (SSYT) 
of shape A with elements taken from {1, • • • , N} is equal to (see e.g., p5|) 

d X (N)= n j _ j 1 = n A n >>■■■ ^ • \ .,. (3.10) 

l<i<j<JV 3 1 j=0 l<i<j<N 

Let D(N) = V v , .V ,:,..,) and OD(N) = Eik^kn Then 
¥(G a (N;a) < n) 

= V{G a (N;a)<n\D(N) = m,OD(N)=k)¥(D(N)=m,OD(N) = k) t 3 - 11 ) 

k.rn>Q 
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Let be the set of N x N symmetric matrices with non-negative integer-valued entries such that 
the sum of diagonal entries is m and the sum of upper-triangular entries is equal to k. For A = 
(ciij)i<i,j<N € In' , the probability that the random matrix (X(i,j))i<ij<N becomes A is given by 



which is a value independent of the choice of A £ I™' k . Hence we have 

F(D(N) = m, OD(N) = k) = - cV^l - g^- 1 )/ 2 ^^™)/ 2 . 



(3.12) 



(3.13) 



Also the conditional probability P(-|D(iV) = m, OD(N) — k) is the uniform distribution on Iff 1 . Now 
we use a version of the Robinson-Schensted-Knuth correspondence which yields a bijection between 
I^f and the set of semistandard Young tableaux with shapes A h 2k + m such that /(A) = m, where 



/(A) = Ax - A 2 + A 3 - A 4 + ■ 



(3.14) 



denotes the number of odd columns of A. Moreover, under the Robinson-Schensted-Knuth correspon- 
dence, the length of the longest up/right path G is equal to the first row Ai. Therefore, we have 

1 



"(G (Y; a) < n\D(N) = m, OD{N) = k) 



\h2k+m 
/(A)=m 



(3.15) 



Xi < n 



Thus from (3.11), (3.13) and (3.15), 



/(A)„(Ai+A 2 +-)/2 



(3.16) 



Ai <n 



where the sum is over all Young diagrams A satisfying Ai < n. Note that if A = (Ai,---Afc) with 
Afe > and k > N, we have d\(N) = 0, and hence the sum in ( |3.16 ) is over the Young diagrams 
A = (Ai, • ■ ■ , Ajv) satisfying Ai < n. Now we use ( 3.1 0| ) and set hj = X n + N — j, and the results (3 
and ( |3.9| ) are obtained. 



□ 



By taking the limit (3J5) of (3^8), we obtain Proposition 1.2 , which was originally given in Remark 
7.6.1, §. 



4 Second formula for geometric percolation : orthogonal poly- 
nomials 

In this section, we present the second formula for the geometric percolation model introduced in Section 
|3|. The following determinant formula is given in Theorem 7.1 of j^] for a more general model. 

Lemma 4.1. (Theorem 7.1, J^J) We have 

P(G (Y; a) < n) = Z 3 (N, a)" 1 E Ue0(n) det((l + aU)(l + ^qU) N ) (4.1) 
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with 

Z 3 (N 7 a) = (l- aVq)- N (l q)-^-^ 2 . (4.2) 

Proof. The random variable G a (N; a) is same as ^(~<f; a) in Theorem 7.1 of ||, where W = Z + and 
"g* = (y 7 ^, • • • , y/q, 0, 0, • • • ) with y/q's occurring N times. (In Section 7 of 0], if is denoted by q, but 
we take this notation to avoid the confusion with the parameter < q < 1 of the geometric random 
variable.) By (7.32) of M, we have 

P(G*(N;a)<n)=F(l% + (lf:,a)<n) = Zl(lf;a) £ a^> SA (/tz + ) (4-3) 

i(\)<n 



where £(X) is the number of parts of the Young diagram A and /(A) is same as (3.14). Indeed this 
formula is, after the modification A i— > A*, is equal to the formula (3.16) in Section^. The normalization 
constant is by (7.11) of H, 

Z 3 (N, a) = Z% + (If; a)" 1 = (1 - ay /q)~ N {1 - q)-^^ 2 . (4.4) 

From (5.55) of [| with the modification as in the paragraph preceding Theorem 7.1 of 0, the sum in 
(4.3) is equal to 

J2 a^ x h x (/lf z+ ) = E ue0(n) det((l + aU)H(U; > /Tz + )). (4-5) 

£(\)<n 

But by (7.28) and (5.6) of @, we have 

H(U; 0*/T z+ ) = E(U; tzJ = (1 + y/W) N \ (4-6) 



and hence we obtain (4.1). □ 

Let 0(n)± denote the connected component of 0(n) with det(?7) = ±1, respectively. The authors 
in H expressed the expected value over the orthogonal group in (4.1) in terms of the related orthogonal 
polynomials. Set 

i>{z) = 1>(z; M, N) := (1 + y/qz) N (l + v^T ■ (4-7) 

Let TTj(z) be the monic orthogonal polynomial of degree j with respect to the measure ?p(z)dz/(2iriz) 
on the unit circle : 



dz 

-Kj(z)-Kk{z)lp{z)—— = NjSjk, (4. 
\z\ = l 2mz 

where Nj is the square of the norm of 7T, (z) . We also set 

7T* = z^jiz- 1 ). (4.9) 

Remark. In general, 7r* is defined by z J Wj(z ). But for the case at hand, all the coefficients of irj are 
real, and hence taking the complex conjugate has no effect. 
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We also set 

faiz) =(l + ^qz) N ir J (z), (4.10) 
cj>*(z) = (l + ^qz) N n*(z). (4.11) 

Note that a version of strong Szego theorem yields that (see e.g., Jl8||) 

lim E ue0 (n) ± det((l + VQU) N ) = (1 - q)-^- 1 ^ 2 . (4.12) 

n— >oo 

From Theorem 3.1 and Theorem 2.3 in g, ([□]) has the following expression. 
Lemma 4.2. For n > 1. 

P(G (7V;a) < 2n) = ^|[^ B _ x (-a) - a^.^-^A"" + [^„_ x (-a) + afc„_i(-a)]A++ } 

(4.13) 

P(G (iV;a) < 2n+ 1) = 5 {[&,(-«) + a0 2 „(-a)]A+~ + [& n (-a) - a0 2 „(-a)]A;+} (4.14) 



(4.15) 



where 



A"" = II ^2i+a(l + (4.16) 



j>n 



A++ = J] ^+ 2 (l - T«+a(0)) (4.17) 



j>n 



A+- = II - ^+i(0)) (4.18) 

A-+ = 1] ^+1(1 + ^+i(0)). (4.19) 



We ako /lave /or n > 1, 



*(JV;l)<2n)=A;+ 
P(G (iV;l) < 2n+ 1) = A++ 



The main results, Theorem 1.3 will be obtained by analyzing the orthogonal polynomials 7Tfc asymp- 
totically. This asymptotic analysis will be carried out in Section [| But we first need the following 
section which will be used for the analysis in Section |^. 

5 Painleve V 

In this Section, we prove various properties of a Ricmann-Hilbcrt problem for Painleve V solution. 
These properties will be used in the next Section rl for the analysis of orthogonal polynomials 7Tfc, and 



also for the proof of Theorem 1.3. This section is, however, independent of other sections. 
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Fix < a < 1. Let Ti = {w € C : \w - 1| = a} and F 2 = {w : \w + 1| = a}. We orient the circle I\ 
counter-clockwise, and orient the circle r 2 clockwise. Set r — T± U r 2 . Let f2i, il 2 and Slo be the open 
regions as indicated in Figure [I]. 




Figure 1: Contours Ti, T 2 and the regions f2i,f2 2 ,i7o 



Let 



<J>( W ) = x, N) 



(l + w) N 
{l-w) N 



e 2- 



(5.1) 



Define the 2x2 matrix V(w) = V{w; x, N) on T by 







$(w)\ 


V(w) = 




< 




1 J 




f i 




V(w) = 




°) 






- 1 V 



io g r 2 . 



(5.2) 



Let the 2x2 matrix M(w) = M(w; x, N) — (Mjk{w))j t k=x,2 be the solution to the following Ricmann- 
Hilbert problem (RHP) : 

M(w) is analytic in w G C \ T, 

M+{w) = M-(w)V(w), weT, (5.3) 
M(w) = I + Oiw" 1 ), as w ^ oo. 
The solution M shares the following properties. 

Proposition 5.1. Set <j\ = (J J) and set 03 = (0 -i)- We have the following properties : 

(i). There is a unique solution M(w) = M(w;x, N) to the RHP ( |5.3| ) for each x > and N € N, and 
the solution M has the expansion 

Mi Mi 

M(w) = I+ — + -4 + ---, w^oo. (5.4) 

ill 



(ii). M(to) = M(w), M(w) = aiM(-w)ai, and detM(w) = 1. 
(Hi). M(w) is real for w € K, and M% and Af 2 have the form 



Mi = 









\-f> 


M 2 = ( 







i(a 2 -/? 2 ) 7 



7 



(5.5) 



-ffere a, /J, 7 are reaZ constants which depend on x and N . 



19 



(iv). We have 



This implies, in particular, 



^-M = ±w[M, a 3 ] + ±0*iM. (5.6) 



a' = \p\ /3' = ^(a/3- 7 ), (5-7) 



where ' denotes the derivative with respect to x, 
(v). We have 

d 1 N N 1 N 1 

— M = -x[M,a 3 ] - 2 Mff 3 + -77- AM - — — — axAa^M, (5.8) 

aw 4 1 — t/r 21 — w 21 + w 

w/iere 

A = M(l)a 3 M(l)- 1 = a 3 + Ijlfi + ^[Sa^-I + Afi) + ^x 7 a 1 a 3 . (5.9) 

('raj. Sei = 6(x;N) = — ^xa(x). Then 6 solves the Painleve V equation 

{x6"f = (9- x6')(6 - x6' + Aid') 2 + AN6'). (5.10) 

(vii). Fix < e < 1/4. For eac/i /irced JVeN, asu +oo, 

/?(*) = -2(-l) JV L^_ 1 ( a; ) e -^ + O^ 1 "^) = g^e"^(l + O^" 1 )) (5.11) 

and 

^ 2L« Mf^dy + 0(e"f = ((jV _ 1)!)2 e-(l + O(x^)), (5.12) 

where L^j^^x) is the Laguerre polynomial of degree N — 1 loitft parameter 1 ; 



4LW=E .,JV = ^/ (1 + -) e 2 (1 + -)- (5.13) 



AT-1 

Also if we take F\ and T 2 to be the circles of radius e, centered at 1 and — 1, respectively, we have 
for each fixed w € C \ (I"i U I^), as x — > +oo 7 

*(.)='-(„ ° m W) ) + 0(e-<-*), (5.14) 

VA(-w,x, TV) / 



where 
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(viii). We have 



M(0) = 1 [ M*) -3 + E n(z) 2 E n(z)- 2 " EnW 2 
2 I En (x)- 2 - E N (x) 2 E N (z)- 2 + E N (x) 2 , 



where En(x) «s defined in (1.2C) with a in (5.5). 

Proof. (i) The proof of (i) is parallel to the analysis of Theorem 5.50 in jl_ 
an outline of the proof. Let C be the Cauchy operator given by 

(Cf)H = [ —ds, f e l 2 (t). 

Ziri Jy s — w 

Let C+ be the limit from the positive side of the contour : 

(C+f)(w) = lim (Cf)(w'), w' is in the (+)-side of T 

w' — >W 

Define the operator Cy acting on L 2 (T) by 

(C v f)(w) := C+(f(I V-'Kw), f e L 2 (L). 



(5.16) 

We here only present 
(5.17) 



(5.18) 



(5.19) 



From the general theory (see e.g. Appendix I, 
the function M(w) defined by 



if the operator 1 — Cy is invertible in L 2 (T) , 



M(w) =1 + 



1 
2~7ri 



(I+(l-C v )-\C v I))( S )(I-V-i)( S ) 



(5.20) 



solves the RHP ( |5.3| ) in L 2 sense. But since V and V~ 1 are analytic on T, the L 2 solution is 
indeed the classical solution of (5.3) (see e.g. Proposition 5.80, |l(|). Just as in Step 1 and 2 of 
the proof of Theorem 5.50, [pL0|] , 1 — Cy is a Fredholm operator of index 0, because the contour 
r is compact and V is real analytic on T. Thus it is enough to show that Ker(\ — Cy) = {0} in 
order to prove that 1 — CV is invertible. Now suppose (1 — Cy)f = for / 6 £ 2 (r). We will show 
that / = 0, which will prove that 1 — CV is invertible. Set 



n(w) 



1 
2~7ri 



ds. 



s — w 

From the properties of the Cauchy operator, n satisfies 



(5.21) 



— n(w) is analytic in C \ T, and continuous up to the boundaries. 

— n + (w) = n-(w)V{w), uET. 

— n(w) = O(^) as w — > oo. 

Let r be the imaginary axis, oriented from ioo to —ioo. Define N(w) by 

1 -$(w) N 
,0 1 
1 



N(w) — n(w) 



w e Sl , Re(w) > 0, 



N(w) = n(w) 
N(w) = n(w) 







-^(w)- 1 1 



w e Cl , Re(w) < 0, 



(5.22) 



Then N satisfies 
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— N(w) is analytic in w e C \ r . 

' 1 

^{w)- 1 , 



- N + (w) = N^(w)Vo(w) for w € To, where Vb(w) 



Now consider 



a{w) := N{w)N(-w) . (5.23) 



Then a(iy) is holomorphic in C \ To, and hence by Cauchy's theorem and the decay property as 
w — > oo, we have 



J a+(w)dw = 0. (5.24) 



But from the jump condition of N, 



a+(w) = N+(w)N-(-w) =N+{w)N-(w) = N+(w)(V (w) ) N + (w) (5.25) 



for w e T. Thus using the property &(w) 1 = &(w) for w e r , 

= y (a + (w)+a + (w f)dw = J N+(w)((V (wf 1 ) T + V (w)- 1 )N^wj T dw 



(T 



(5.26) 



I 2 )N+(w) dw. 



This implies that (Ni 2 ) + (w) = (^V 22 ) + (u>) = for almost every w & T, and hence by Cauchy's 
theorem, we have 

N 12 (w) = N 22 (w) = 0, i?e(u;)>0. (5.27) 
This in turn implies, from the jump condition, that 

Nu(w) = N 2 i(w) = 0, Re(w)<0. (5.28) 
On the other hand, from the jump condition again, 

(iY n ) + H = (iV 12 )_H e ^-f " " 

Set 



(JVii)+(w) = (N 12 )_(w)ei xw l ) , ^e*. (5.29) 



Ja + H^nM, i?eH>0, 
:= < (5.30) 
[(l-w) N ei xw N 12 (w), Re(w)<0. 

Then h is entire, and h(w)w~ N ~ 1 — > as w — > oo since N(w) is bounded for w G C. Thus by 
the Liouville's theorem, h(w) is a polynomial of degree at most N. But then for Re(w) < 0, 
N 12 (w) = h(w){l - w) N e-2™ blows up as an oo unless h = identically Therefore we obtain 

N 12 (w)=0, Re(w)<0, 

(5.31) 

N n (w) = 0, Re(w) > 0. 
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By a similar argument, we obtain 

N 2 i(w)=0, Re(w)>0, 
N 22 {w) = 0, Re(w) < 0. 

Therefore we have N(w) = and hence n(w) = for all w E 1 
n+ = C+{f(I - V- 1 )) = CvU) = f, we obtain that Ker(l 



(5.32) 

Since by f[l|) and 
Cy) has dimension 0. The 



uniqueness of the solution is standard. The expansion (5.4) follows from (5.21). 



(ii) The jump matrix V have the properties, V(w) = V(w), V(w) = <t{V{— w)~ cri and det V(w) ~ 1 
for w G T. These properties, together with the uniqueness of the solution to the RHP, imply the 
results. 



(hi) The realness of M(w) for w € R follows from M(w) — M(w), which also implies that M± and M 2 
are real. By expanding the both sides of M(w) = aiM(—w)ai as w — ► oo, we have M\ = —<j\M\o\ 
and M 2 — o\M 2 <j\, and hence 



Mi = 







M 2 = 1 


\-t> 








(5.33) 



for some a, /?, 7, (M 2 )n. Also since = logdetM = tr log M = trlog{I + ^ + ffi -\ ), 

as w — > 00, we have £r(Mi) = and tr(M 2 — l-Mf) = 0. The second identity implies that 
(M 2 )n = \{o? — P 2 ) and we obtain the results. 



(iv) Set / := Me" 1 " 3 . Then 



/ is analytic in 

/+ = f-Vf(w) 



:\r, 

for nGf, 



(5.34) 



I 



00. 



where 



v f (w) 




w e Ti, 



w e r 2 . 



(5.35) 



Note that the jump matrix does not depend on x. Thus /' and / satisfy the same jump condition, 
and hence f'f^ 1 is a entire function, where /' = -§^f- On the other hand, from the condition 
fei xwa3 — ► I as w — > 00, we obtain f'f^ 1 + \wfcr3f~ 1 — > as tu — > 00. But since M = 
7+ ^ + 0(u;- 2 ) as w -> 00, we have jwf^f- 1 = ±w 3 + \[Mx,a z ] + O^ 1 ). Thus by 
Liouville's theorem, = — \wcr3 — j [Mi, 173], which is equivalent to 



(5.36) 
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as desired. 

By taking the limit x — > oo in ( |5.6| ) and collecting the terms of order 0{w~ 1 ) 1 we have 

M[ = l[M 2 , <r 3 ] + \$<JiM x . (5.37) 



This implies (5.7). 

(v) Set h = Me-i xu " 73 (^)^ Na3 , where (f^j)' is defined to be analytic in C\ [-1,1] with the 
condition that it becomes 1 as w — iy — > satisfying y > 0. Then 

/i is analytic in C\ ([-1,1] UT), 
ft+ = (-l)^_, we (-1,1), 

V V 1— if / 5 

where the interval [—1, 1] is oriented from the right to the left, and the jump matrix is given 

by 



(5.38) 



v h = I o I 1 



Vh 



1 

1 1 



w e Ti, 
w g r 2 . 



(5.39) 



Then 



M- 1 = MM" 1 - -xMctsM- 1 + — Ma 3 M~ 1 
4 1 — itr 



(5.40) 



is analytic inC\{ — 1,1} with simple poles at —1 and 1, where h denotes the derivative with 



respect to w. Thus, since hh 



— jX(J3 as w — ► oo, we find that 



hh' 1 



1 



7x0-3 



Bo 



1 — w 1 + w 



(5.41) 



with some constant matrices Aq and Bq. By taking the limits of (5.4C) as w — > 1, —1, we obtain 
A = ^Af(l) ( 7 3 A/(l)- 1 and B = f M(-l)er a M(-l)- 1 . Since M(-l) = criM(l)(Ti from (ii), we 
have i?o = —<tiAq<t\. 

By combining ( 5.4d|) and ( 5.41 ), we obtain (5J5) with constant matrix A = M(1)ct3M(1) _1 . Now 
we take the limit w — * oo to ( |5.8| ). By collecting the terms of order 0(w^ 1 ) and noting that 
[Mi, 03] = — 2/3(Ti, we have 

A + a x Aai = —a^oi, (5.42) 

and by collecting the terms of order 0(w~ 2 ) and noting that [M2,<73] = 270-103, we obtain 

2 1 1 

A - o x Ao x = 2(73 + —Mi + —x(3o 1 M 1 + —xyo^. (5.43) 



These yield the second formula (5.9) for A. 
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(vi) From the second formula (5.9) of A, we have 



A = 



N 2N 



± a - J- x 3 2 

N u. 2N^' J 

jjxf3{l + a) + 2^x7 



jV 2N X @(^ a ) 2N X ^\ 



(5.44) 



On the other hand, the first formula, A = Af(l)cr 3 M(l)" 1 , of (|5.9|) implies that det A = -1. Thus 
([5.44D yield the identity 



-(JV - a - ^/3 2 ) 2 - ^ 2 /3 2 + 09 + ^(a/3 - 7 )) 2 = —N 2 . 



By removing 7 and j3 using (pj), we obtain 



.f N _ Q _ ^ a ')2 _ 1 2 a / + J_( 2a ' + xa "f = -N 2 . 
2 2a 



(5.45) 



(5.46) 



This becomes ( 5.10 ) if we set 9 



\xa. 



(vii) We take V as the union of circle of radius e, centered at 1 and — 1. (We have freedom to pick a 
contour.) Then from the formula of V, it is direct to check that 



L°°(r) 



< 



(2+1) 



N 



-(l-e)x/2 



Also by (|5T9| ), 

l|CV||L 2 (r)^L 2 (r) < l|C+llL 2 (r)^L 2 (r)l|J" - ^^IIl^e") < ci||J" - T^IU^r) 



(5.47) 



(5.48) 



for some constant c\ > 0. Here ci can be taken to be independent of0<e<l/4 from a simple 
scaling argument. Hence for large enough x, ||(1 — Cv)~ 1 \\l 2 (t)^l 2 (D < 1/2, and from ( [5.20| ), 

,irr 



M(iu) = I + 



2?ri 



|£--V JLl°°li ) 
dist(w, r) 



(5.49) 



where |T| is the size of T. Also, we have 



Mi 



1 

27ri 



(/ + (1 - C v )-\CvI)){s){I - V- l ){s)ds 



(5.50) 



- ^- y r ((l - C v )-\C v (C v I)))(s)(I - V-^ds, 
and the second and the third integrals on the last line are of order 



oOli-y- 1 !!!^)), 



0(l|/-^- 1 Hi- ( r)). 



(5.51) 



respectively. Therefore, we obtain 

= (Mi)i2 - J (I - V _1 )i 2 («)<fa + 0(||/ - V 



— 1 11 2 \ 
L°°(r), 



1 

2?ri 



l»-l|= 



/V 



(5.52) 



e-2 xs ds + 0{e 



-(l-e)w\ _ 



)=:/ A r(.T)+0( e -( 1 - e ) a: ). 
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After the change of variables \x{s — 1) = z, we have 



2ni 



X-.N-1 



1*1=4* 



z~.dz 
x z 



(5.53) 



This is precisely an integral representation of the Laguerre polynomial It^_y(x) (see e.g., |i 
22.10.8), and we find 



13 = 2{-l) N e-? x L^ ) _ 1 (x) + 0(e-^- £ >). 
Similarly, from ( |5.5C ), we have 

a = (Mxjaa = ~ J((CvI)(s)(I - V^s^ds + 0(\\I - V^Wl-p)) 

<ls I ^^^1 + 0{e -V2d-e) X) = . gN{x) + 0{e - m{ l-e) X y 



(5.54) 



(5.55) 



t-8 



By a direct computation using (5.1), we find that 



9n(x) 



2{2mf 



ri 



ds / dt 



1-t 
T+t 



N 



1 + 8 
1 - S 



e 



-lf N (x)f 



Since lim^^oo §n(x) = 0, we find that 

9n(x) 



X \{.fN{y)) 2 dy, 



(5.56) 



(5.57) 



which proves (5.12). 

For w G C such that dist{w, V) > |e, the result fl5.14| ) follows from ( [5.490 and fl5.47| ), fl5.48| ). For w 
such that dist(w, T) < ^e, we algebraically transform the RHP so that the contour V are now the 
unions of circles of radius 2e, centered at 1,-1, and then apply the same estimates. By undoing 
the algebraic transformation and using the Cauchy's theorem, we obtain the result (5.14). 

(viii) From the properties (ii), M(0) is of the form 



M(0) 



'A(x) B(x) 
.B(x) A{x) / 



(5.58) 



for some real function A and B such that A — B = 1. From the differential equation (5J3) when 
w = 0, wc find 



A' = 1 -PB, B' = l -f3A. 



Thus we have 



A(x) + B(x) 
A(x) - B{x) 



{A{x Q )+B{x ))e^^ {s)ds , 
(A(x )-B(x ))e-^o^ ds . 



(5.59) 



(5.60) 
(5.61) 
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The asymptotics (5.11) and (5.14) then yield that 

A(x)+B(x) = e^^ ds , 
A(x) - B{x) = 

which imply ( 5.1C| ) 



(5.62) 
(5.63) 

□ 



In Proposition 5.1 (i), the solution to the RHP exists for x > 0. Indeed it can be shown that the 
solution ceases to exist when x = for N S N. 

Lemma 5.2. There is no solution to RHP ( jo~3] ) when x — for N 6 N. 

Proof. When x = 0, *(to) = (j^ ) W - Suppose there is a solution M to the RHP (Q. Let L(w) be 
the matrix defined by 



L(w) 



M{w) 



M{w) 



[M(w), 



,0 1 

1 i) ' 

( l—w \N i 



w e Hi, 



(5.64) 



Then L(w) is analytic in C except that its second column has a pole of order N at w = 1 and its first 
column has a pole of order TV at w = — 1. Since L(w) — ► 7 as it) — * oo, we find that L has the form 



c k CP 
.d k 0/ 



for some constants aj,bj,Ck,dk- From (5.64) and (5.65), for w S Ox , 



M(iw) = L(w) 



,0 1 



and especially the 12-entry is 
M 12 (w) 



N 



N 



1 + W,N x ^ dj r^Cn(l+ti)) ,f fc 

1 ^ ^ (1 - wV + ^ (l-w) N ' 
j=l y ' fe=l v y 



1 — w ' 



(5.65) 



(5.66) 



(5.67) 



From ( |5.3| ), M, hence M12, is analytic at w = 1. By expanding (5.67) as a Laurant series around w = 1, 
and computing the coefficient of (1 — uj) -1 , we obtain the condition 



(-l) JV 2jV + ai - (-l) iv Cl =0. 



Similarly, for w G f^2, 



, , v— * dill — W) J \ - 

^N = i + E %r4v + Ett 

7=1 



fc=l 



(1 + w) k ' 



(5.68) 



(5.69) 
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and by expanding M n (w) as a Laurant series around w = — 1, and calculating the coefficient of (l+w) 1 , 
we obtain the second condition 



(-f) jv ai + Cl = 0. 



(5.70) 



But there is no such ai,c\ satisfying (5.68) and (5.70) when N 6 N, and this is a contradiction. 
Therefore, there is no solution M to the RHP (5.3). □ 



6 Asymptotic analysis of orthogonal polynomials and the proof 
of Theorem 



Set 



(6.1) 



As mentioned earlier, Theorem i.3 is obtained by taking the limit (3.1) of the formula Lemma |4.2| . For 

we set 

(6.2) 



that purpose, we need asymptotics of TTk and N k - In view of Lemma p.l| and Lemma 4.2 



t = 1 , k = \xL], 



and take the limit L — > oo in this section. The results are summarized in Proposition 6.4 and Proposition 
6.6 below. The asymptotics of A^* are in Proposition 6.5, and the proof of Theorem 1.3 is given at 
the end of this section. 

Let £ be the unit circle {\z\ = 1} in the complex plane, oriented counter-clockwise. Let Y be the 
solution to the following 2x2 Riemann-Hilbert problem (RHP) : 

Y(w) is analytic in w G C \ S, and continuous up to the boundary, 
1 w- k (l +tw) N (l +tw- 1 ) N ' S 

V° 1 

Y(w)z- ko ' 3 = I + Oiw- 1 ), as «n oo. 

Then due to the work of Fokas, Its and Kitaev (|12 , see also B), the orthogonal polynomials irk and 



Y+(w) = Y_(w) 



w £ S, 



(6.3) 



its norm of (4.8) are given by 

ir k (w) = Yn(w), 



N k = Y 12 (0), N k ^=-Y 21 (0y 



(6.4) 



The goal of this section is to find the asymptotics of Y with precise error bound as L — > oo with ( |6.2j ) , 
and hence to find the asymptotics of TTk and N k - We use the steepest-descent method for RHP, which 
was introduced by Deift and Zhou [ pd| . Throughout this section, N is a fixed parameter, while t and k 
would vary as L varies. 
Define by 

v o (i + tz)- N r ' li o 



(z) = (-l)^Y(z) 



-1 



\z\ < 1, 



(6.5) 
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and by 



m (1) 0) = (-l)' CT3 r(z) j 
Then rrv 1 ' solves the new RHP 



z- k {l + tz- 1 ) 



1\N 







z k {\ + tz~ x ) 



l\-iV 



(-1)" 



\z\ > 1. 



(6.6) 



m± (z) = m_ [z ) 



1 -<p(z) 

Viz)' 1 
(z) = / + 0(z^ 1 ), as z — > oo, 



(6.7) 



where 



:= (-z) fe (l + tz) JV (l + iz- 1 ) 



1\-JV 



(6.8) 



This RHP is algebraically equivalent to the RHP ( |6 . 3|) . 

Fix < a < 1. Let E x = {\z + 1 - i| = ^} and E 2 = {|z + 1 + ^| = ^}. We orient the circle 
Si counter-clockwise, and orient the circle E2 clockwise. Note that we have plenty of freedom for the 
choice of the contour. Since < a < 1, Si, E2 an d \z\ = 1 have no intersection, and the complex plane 
is divided into four connected regions (see Figure |). When L > which we assume hereafter, the 




lzl=l 



Figure 2: Contours Si nd £2 
point — t" 1 is inside the disk bounded by £2. Define rnP'{z) by 

1 v{z) 



m< 2 )(z) = m«(z) 
m^{z) = mP-'{z) 



1 
1 

V^)- 1 1 

m( 2 )(z) = m^{z), for z inside Si and S2 



for z between Si and \z\ = 1, 

for z in the unbounded component, 



(6.9) 



Since <fi(z) is analytic in C \ {— t}, and (p(z) 1 is analytic in C \ {— t 1 }, we find that (z) is analytic 
in all four regions. Moreover, by the jump condition of on \z\ = 1, vnS 2 ^ is analytic on \z\ = 1. 
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Thus m/ 2 ) solves the following RHP 



m {2 \z) is analytic in C \ (Si U E 2 ), 



^\z) = m^ ) {z)v^(z), zeSiUE 2 



where 



v^(z) = i 



Now we take the scaling 



,( 2 ) 



l + 0(z- 




as z — > oo, 










<p(z 


z G Ei, 




I 


1 


















-1 





(6.10) 



(6.11) 



:= 2L(z + l). 



(6.12) 



Under this map z i— ► iu, Ei and E 2 are mapped to Ti := : |w — 1| = a} and T 2 := {u> : |tu + 1| = a}, 
respectively. Set r — T± U T 2 . We define 

to 



Then solves the RHP 



m (3) (w) :=to( 2 )(-1 + — ). 



?/ 3 ) (iu) is analytic in w € C \ T, 



(6.13) 



(6.14) 



i( 3 )(z) =I + 0(w- 1 ), as 



u> — > oo, 



where 



;(3)( u ,) =m (3)(_l + ii). 



(6.15) 



Now we need the following Lemma. 



Lemma 6.1. For fixed < a < |, where a is the radius ofT± and T 2 , we have the following results. 



(1) With (6.2), for any Xq > 0, there are positive constants C\,C\,L\ such that 



M-l + ^) - ^HIU- (ri ) < ^~ ClX i M-l + ^) - *HIU-(r a) < ^~ Cl * (6.16) 
/or aZZ L > L% and x € [xo, oo). 
(i2) .For any Xq > 0, there is a constant C 2 > such that 



M 



+l|L«>(r) 



<C 2 , HM+^liao/r) < C 



(6.17) 



/or a: £ [a^o, oo). 
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Proof. (1) By an elementary calculation, we have for any complex numbers /, g 

\e f -e 9 \ < \ f - g\ e max( - Re W> R < g » . 



For weTi, with fl6.2|), 

|v(-i + 2i)-*H 

Setting 



(1 



2L ) 



e »" 



1 + w 



l — io 



iV 



[zL] + TV to 
/= ^^ l0g(1 -2l } ' 



2l 



for (6.18), the difference (6.19) is less than or equal to 

,2 + a.N 



e xLf _ e xLgl 



(6.18) 



(6.19) 



(6.20) 



(6.21) 



Now with fc' 



-^j— and e .- j C , 



f-g = k'log(l-e) + e= f ^ (k' log(l - se) + se)ds 

Jo ds 

- 1 -(fc' - l)e - se 2 + (fc' - l)see + s 2 e 2 e 



(6.22) 



|l- S£ p 



-ds. 



An elementary calculation yields that \arg(w)\ < ^ for w G Ti as < a < |, and hence we have 
|arg(e)| < | and |ar 5 (e 2 )| < f . Also, when L > > , we have ± < |1 - se| 2 < 2 for 

hi e Ti. Hence noting that k' — 1 > 0, we obtain 



Re(f -g)< I {-Uk 1 - l)Re(e) - \sRe{e 2 ) + 2{k' - l)s|e| 2 + 2s 2 \e\ 2 Re(e)}ds 
Jo 2 2 

= -\{k> l)ife( e ) - ii?e(6 2 ) + (fc' - 1)| £ | 2 + |e| 2 i?e(e). 



(6.23) 



The first term satisfies —\{k' — l)Re(e) < 0. For the third term, when L > ^f^-, we have k' — 1 = 
< ^ < i, and hence we find (fc' - l)|e| 2 < \Re{e 2 ) as |ar 5 (e 2 )| < f . It is also direct to 
check that the forth term has the estimate ||e| 2 7?e(e) < j^Re(e 2 ) if we take L > These estimates 
yield 



Re(f - g) < —}-Re(e 2 ) < 
16 



(6.24) 



when L > max(#,i^) 

— ^ 3 ' XQ I 



Similarly, from ( |6~2^ ), using |e| < < f and |1 - se\ 2 > \ when L > we obtain 

|/ - , 9 | < 2 /'{(fc' - l)|e| + S | £ | 2 + (fc' - l)|e| 2 + S 2 \e\ 3 }ds 



AN . 1 ,47V 16 N 1 

Xq Xq 3 L a 

<3(^ + 4)- 1 
x 



(6.25) 



L 2 ' 
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Now the first inequality of ( |6.16 ) on Ti is obtained using (3.21). The second inequality of ( |6.16 ) on T2 
follows from the inequality on Ti and the symmetry under w 1— » —W. 

(2) From ( p5~2Cl ) and ( p^|) , we have M+ = I + (1 - C V )~ X {C V I). By Proposition (1 - C v )~ x 
exists for all x > 0, and also it is easy to check that Cy and (1 — Cy) -1 are continuous in x. Hence 
from ( [5~2(i| ), M + is uniformly bounded for a; in a compact subset of (0, 00). Also, from the analysis of 
Proposition 5.1 (vii), Cy and (1 — Cy)' 1 are indeed bounded as x — > 00. Hence we obtain the uniform 
boundedness of M + for x € [2:0,00). The boundedness of M7J follows from the boundedness of M + , 



together with the fact that detM + = 1. 
Set 



Then R solves the RHP, 



R(w) :=m^'>(w)(M(w))- 1 . 



R+(w) = R_(w)v R (w), ioeC\S, 



R(w) =I + 0{w~ r ), 



as w — > 00, 



□ 



(6.26) 



(6.27) 



where 



v R = M_t) (3) V'~ 1 (M_)~ 1 = M + V- 1 vWm+ 1 . 
Then Vr shares the following properties. 



(6.28) 



Lemma 6.2. Let C VR be the operator introduced in (5.19) for vr. For any fixed xq > 0, there are 
positive constants 63,03,-^3 such that 

C3 -c,X 



I-v 



R ||i°°(r) 



< 



(6.29) 



uniformly in x £ [xq,oo) for L > L3. Also for L and x in the same range, the operator C VR acting on 
L 2 (T) satisfies 



\(1-C VR ) 1 \\t2(r)->L 3 (r) < 2, 



(6.30) 



and the matrix R in (6.26) is given by 

R(w) = m {3) (w)(M(w))- 1 

I - vb(s)- 1 



= I 



1 
2~7ri 



ds 



2ni 



((1-C Vr )-^C Vr I)(s)(I-vr(s)- 1 ) 



ds 



(6.31) 



Proof. Since 



(6.32) 
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for w £ Ti, from Lemma 3.1 (1), we obtain for L > L\ 



\\I-{v^V\\ L ~ (Vl) <^-e 



(6.33) 



A similar calculation yields the same bound for IV Therefore using Lemma 6.1, we obtain for L > L\, 

1 



II-v^lU-CD = H M +( J - (« (8) )" 1 ^)^ 1 IU-(r) < 2C x C^e-^, 



(6.34) 



and we obtain (6.29) (with C3 = c\). For ( j6.30|) , note that since the Cauchy operator C+ on L 2 (T) is 
bounded, we have 



c 

l|CuRlk 2 (r)— L 2 (r) < -j-e 



(6.35) 



for L > L\ with some new constant C > 0. But if we take L large enough so that 5-e C3X < 5- < i, 



we have 



||C«alU 2 (r)— £2(r) < 



(6.36) 



uniformly in x S [20,00). Now the result for C VR follows from the Neumann series, and fl6.27D follows 



from the general theory of RHP (cf. proof of Proposition 5.1 (i)) 



□ 



Under the sequence of transformations Y — > nrS 1 ^ — » m/ 2 ) — > m^ 3 \ the quantities (6.4) of interest 



become 



N k = m^(2L), iV-_i x = mg ) (2L), 7^(0) = -(-l) fe mg ) (2i). 
From (6.31), using ( |6~29| ) and ( |6~30| ), we find that for x, L as in Lemma I6.2L 



|m( 3 )(2L)M(2L)- 1 - 71 < %e~ c ^ , 



(6.37) 



(6.38) 



for some constant C" > 0. 



Now the following estimates for M(2L) follows from ( 5.20 ) in the proof of Proposition 5T (i) and 
also the proof of Lemma |6.1| (ii). 



Lemma 6.3. For any fixed xq > 0, let C3 and L3 be as in Lemma 6.i. There are positive numbers C4 
and C5 such that 



\M(2L)-I- 



Mi, (h 
2L 1 " L 2 ' 



\M{2L)\<C 5 , 



(6.39) 



for all x € [xq, 00) and L > L3. 



Thus (|6.38D yields that 



| m (3)(2L)-J-^|<g e -- 



(6.40) 



for some constant Cq > 0. From this and (6.37), we obtain the asymptotics of Nk and 7Tfc(0) : 
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Proposition 6.4. With (6.2), for any fixed xq > 0, there are positive constants Cq,cq,Lq such that 



N k -1 + 



K-i - 1 - 



a(x.N) 


< 


c 6 _ 


-c 6 x 


2L 






5 


a(x,N) 


< 


c 6 _ 


-c 6 x 


2L 




v e 


5 




< 




-C 6 X 


2L 









(6.41) 
(6.42) 
(6.43) 



where a and (5 are defined in Proposition 5.1 (Hi). 



As in the proof Lemma 7.1 of it is direct to obtain the following result from Proposition 6.4, and 
we skip the proof. 

Proposition 6.5. For any fixed xq > 0, there are positive constants Lj, cj such that when 2n = [xL], 



11^+2 = exp( r l a (y,N)dy)(l + 0(je- c ^)), 

j>n \Jx * / 

n^+l = exJ [°° la(y,N)dy)(l + 0(je-^)), 

j>n VJl ' 

n(l±^ +2 (0)) = expf T f \p(y,N)dy){l + 0(je- c n), 

j>n V Jx ' 

n(l=F7ny+i(0)) = exp(r J™ \p(y,N)dy)j(l + 0(±e- c n), 



(6.44) 
(6.45) 
(6.46) 
(6.47) 



for x £ [xq, oo) and L > Lj. 

Also under the transformations Y 
z = — 1 + given by 



,.(2) 



m( 3 \ (f>k in (|4.10|) is, with the relation 



-(-i) k m$(w), toeOi, 
<f>k{z) = { z k {\ + tz) N {l + tz- 1 )- N mf 1 \w), w e Q 2 , 

-(-l) k rn$(w) + z k {\ + tz) N {l + tz- 1 )- N m ( ^ > (w), w e fl 

(recall Figure |l|). Hence, under (|6.2|), from ( |6.31 ), for any fixed x > and fixed aieC, 

lim m (3) (w) = M(w). 

L — >-oo 



(6.48) 



(6.49) 



(For w near the contour T, we could take different radius a of the contours Fi,r2.) Therefore, ( 6.48 ) 
has the limit 



lim (-l) fc ^(-l + — ) = { -M 12 (w) + $HM n H, w £ Q , 



w e Hi, 
w e ri 2 , 



(6.50) 
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for each fixed x > and w G C, where k = [xL\. On the other hand, by noting that (j>%{z) 
z k {l + y/qz) N (l + ^/qz- 1 )- N (f) k (z- 1 ), we obtain from flsliol) that 



Afn (-to) 

^lim^-l + ^-) = <j M u (-«;) - $(w)M 12 (-w), 
-$(to)M 12 (-tu), 



w g f?o, 

10 G ^2- 



Using the symmetry M(ui) = ctiM(— w)o~i of Proposition 5.1 (ii), this is equal to 



M 22 {w) 

lim <p* k (-l + = I M 22 (w) - $(n)M 2 iH, 



2L 

Define M^\w) = M^(w;x,N) by 

' M{w) 



-$(w)M 21 (w), 



w € Oi, 
to € Oq, 
u> G £l 2 . 



Af«( 



M(w) 
M(w) 



1 -&(w) 
,0 1 7 

1 -$(w) 
^(w)- 1 



w G Ooj 
iog0 2 \{-1}. 



(6.51) 



(6.52) 



(6.53) 



In other words, from the jump condition of the RHP ( |5.3| ), is obtained by taking analytic con- 

tinuation of M(w) for io G ili. Then the results ( 3.5C| ) and ( |6.52 ) can be written in a compact form 



Proposition 6.6. We have with k = [xL], 



lim (-1)^-1 + _) 

L — *oo ALi 



l, ■ - xd ^ 2,L 



lim 



(6.54) 
(6.55) 



for each x > and w G 



Proof of Theorem 1.3 



Now we prove Theorem 1.3. By combining Lemma p.l[ Lemma 4.2, Proposition 6.5 and Proposition 
3.6 , we obtain for p > and x > 0, 

F{H®(N;p) < x) 

lfr..m, , ..m , r ..m, , ..in L ( 6 - 56 ) 



-\ [M£(w) - <(«,)] (En^))" 1 + [<(«)) + M{^H] E N (z) ^ F N (z), 



where 



10 = 1. 

P 



(6.57) 
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Thus we set 



2 2 
a N (x,p) = M 2 ( 2 } (- - b N (ar,p) = M$(~ - 

P P 



(6.58) 



From the RHP for M, a,jsi(x, p),b;si(x, p) are analytic in x > 0, p > 0, and En (a;), Fn(ze) are analytic in 
a: > 0. 



The properties (i), (ii) of Theorem 1.3 are given in Proposition 5.1 (iv), (vi) and (vii). 

For the property (iii), we note that it is direct to check that when M satisfies the differential 
equations (vi), (v) of Proposition 5.1, also satisfies the same differential equations. This implies 

the property (iii). 

The asymptotics (iv) of aN,t>N follows from (5.14), and the definition of 

The asymptotics ( 1.31 ) in (v) follows from the normalization condition of the RHP : M(w) — > / as 
w — > oo, and ( 1 -32j ) is obtained from Proposition 5.1 (viii) and the definition of 
Now we compute ( 1.35 ). With x = yp and w = ~ — 1, we have 



$(w; x) = e v e 7 - 



yp. 



(P-I) 



N 



0(ei yp ). 



As p — > oo, using ( 1.2q ), with the change of variables s = 1 — (2/ p)u, 

Mvp,p) - *(- - i;»p)(ri(-- + i,yp) + o{e-^-^y)) 
p p 



i 

1 

2ni 



,v(u-i) ( P-U\N 



du 



\u\=l/2 



\u\ = l/2 



u N (u-l) 



0(e 



<i-2e)yp 



-^- Ty (l + 0(^ ) )+0(e-(i- 2 ^). 



But the function 



Av) 



l 

2ttI 



,y(u-X)_ 



du 



\u\ = X/2 



u^(«- 1) 



(6.59) 



(6.60) 



(6.61) 



iV — 1 

satisfies g \y) — e~ y y N _ 1 y an d .9(0) = 0. Hence g(y) = P(N,y), the incomplete Gamma function, and 
we obtain the first of (1.33|). The second of (1.33) follows from (1.29) and (6.59). 



7 Limiting distributions as TV — ► oo 

In Theorem 4.2 of the authors computed the limiting distributions of the last passage percolation 
time G (iV;a) of (3.4) as N — > oo for various values of a. Similar results are also obtained in |J for 
a Poisson percolation model with a symmetry condition, which also has interpretations as the longest 
increasing subsequence of random involutions. In this section, we take similar limit for H a (N; p). The 
results are such that we take the formal limit Lemma 3.1 in Theorem 4.2 of assuming that the two 
limits N — > oo and L — ► oo interchange. The functions Fx, i*4 and F are defined in |M pL 
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Theorem 7.1. For each fixed i€l and p, 



lim ] 



H a {N;p)-AN 
2 4 / 3 iV 1 /3 



< x 




Also, for each fixed x G 



lim 



and t«el, 

H a {N;p) -AN 
2 4 /3JV 1 /3 



< x = (!;«;), 



< p < 2, 
P = 2, 
p > 2. 

2 5/3, 



(7.1) 



ATi/3 



(7.2) 



From the formula in Theorem 1.3, this result can be obtained by applying the Deift-Zhou steepest- 
descent method to the Riemann-Hilbert problem (5J3). The analysis is analogous to that of |Q, q], 
and we do not provide any details here. Steepest-descent analysis for an extension of the RHP (|5.3|), 



which includes the analysis of the above Theorem 7A as a special case, will be carried out in a later 
publication, [||. 



8 Correlation functions 

In this last section, we give some remarks on the correlation functions for the general interpolating 
ensemble with the density ([ll]) with /3 = 1, 

1 N 



^ A l<i<j<N j=l 



pJV 



„ ord :— {£jv < Cat-i < • • • < ?i} (N is even). It has been known that the correlation functions for 

the orthogonal ensemble (when A — 0) and the symplectic ensemble (when A = +oo) can be expressed 
in terms of the Pfaffian, or the square root of the determinant, of an antisymmetric matrix (see e.g., 



j22L |29|). For (8J) with general A, Rains (|23j]) computed the correlation functions and showed that it 
is again expressible in terms of Pfaffians but of a different matrix. In this section we remark that the 
result of Rains can be obtained from the argument of Tracy and Widom |^| after a minor change. 

In [ p9[ , Tracy and Widom developed a systematic method to express various correlation functions of 
orthogonal and symplectic ensembles in terms of certain Fredholm determinants. Especially in Section 



9 of 1 29 1, Tracy and Widom expressed the correlation functions of general orthogonal ensemble. The 
computation below for the above ensemble ( |S.l|) is identical to that of Section 9 of [^9| except the change 
of the asymmetric factor e — > ea- 

We assume that N is even in the below. When N is odd, one needs some change of the formulas. 
The starting point is the following identity in Remark 7.6.1, § : 

e A£» l( -D^ = Pf(sgnte _ ftje^-fcl)^^. (8.2) 

This can be checked by noting that the Pfaffian on the right-hand-side is the square root of the 
determinant of the matrix (sgn(£j — ^') 1<j KA n an d by finding the relation Dn = 
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e 2A ^ N - 1 5™)£)jv~2 using proper row and column operations. From (8.2), for any bounded function 
/, we have 

N 

Za- I p(&,--- .^Ai^na + ZfeMi-^ 

J=! (8.3) 

= / Pf(sgnfe - & ) e >i|Ci-&l) det(C>(a)(l + /(&))) x<j fc<JV #i ■ ~d£ N 

We also need the following result of de Bruijn (equations (4.6)-(4.8) of ||) : given a measure space 
(X,/j,), for an anti-symmetric function s(x,y) = —s(y,x) on X x X, 

— [ P{(s(xi,Xj))i< i; j< N det(<l>i(xj))i<i,j< N dfi(xi)---dii(x N ) 

iV! / 

fff \ (8 - 4) 

= Pf / / <j> i (x)s(x,y)^j{y)dfj,(x)d li (y)\ 

\JX JX J l<i,j<N 

for a sequence of functions <f>j on X. This is a generalization of (1.4) of |2{J where the authors took the 
special case when s(x, y) — sgn(x — y). Now with s(x, y) — sgn(x — y)e A)[X ~ y][ and (f>j(x) — x^w(x)(l + 
f(x)), the square of ( |S.3|) is equal to 

detf / / sgn(x - y)e A ^x^y k w(x)w(y)(l + f(x))(l + f(y))dxdy) . (8.5) 

\Jm.Jm. / o<j,k<N-i 

Here the N\ term in (8.4) disappears since K^. d is the ordered set {£jv < ■ • • < ?i} and we take X = E. 
This formula is same as the second displayed formula of section 9 of 29 with the modification that 
e(x — y) (which is sgn(a; — y)) is replaced with sgn(x — y)e A)[X ~ y][ . 

The rest of argument is same as section 9, [ p9[ except that the operator e whose kernel is sgn(x — y) 
in p{| is now changed to the operator ca, defined by 

(e A h)(x)= I e A (x-y)h(y)dy, e A (x - y) := sgn(x - y)e A \ x ~ yl , (8.6) 



for a suitable class of functions h. With this modification, the result analogous to (3.3), (9.1) of pj| is 
the following. 



Lemma 8.1. (Theorem 1.1 of \23\j) Set tpj(x) — Pj(x)w(x) where Pj(x), j = 0, 1, • • • , is an arbitrary 
sequence of polynomials of exact degree j. Let M be the matrix 



M =([ [ e A (x - y)iij(x)i; k (y)dxdy) (8.7) 

\JRJR / 0<j,k<N-l 

and set M^ 1 — (/J,jk)- Then we have 

r N I 

J p(£i,-~ ,£NlA)]l(l + f(Z j ))d( 1 ---dZ N = y/tetQ. + Kg>f) (8.8) 



(A) 

where the operator K N has the 2x2 matrix kernel 



v , A s ( S N {x,y;A) S N D(x,y;A)\ 

K N (x,y;A) = (8.9) 
\IS N (x, y; A) - e A (x - y) b N (x,y;A) I 
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AT— 1 

S N (x,y;A) = - ]T ij)j(x)njk{e A ipk){y), (8.10) 

j,k=0 
N-l 

IS N (x,y;A) = - (e A tp j )(x)n jk (e A ip k )(y), (8.11) 
j,k=o 

N-l 

S N D(x,y;A) = ^ ^{x^jMv)- ( 8 - 12 ) 

3,fc=0 

The matrix elements in the above determinant for general A is significantly simplified for the Laguerre 
case, w{x) = e~ x l x >o by jl5|], which is an extension for general A of the results for Laguerre orthogonal 
and symplectic ensembles (see e.g. |p[ |3lf). 

When A = 0, e A = e in the notation of p9[ , and the kernel ( |8.9| ) is equal to (9.1) of |29|], which 
is the (3=1 orthogonal ensemble. On the other hand, when A — > +oo, we do not have a proof that 
Kn(x, y; A) converge to the kernel (8.1) of pj| for the (3 = 4 symplectic ensemble. However, we note 
that for smooth h which decays fast at ±oo, integrations by parts yield that 

(e A h)(x) = -^h'(x) + ±(e A h")(x) 

\ 2 , (8.13) 

= -tfXW ~ T , hm ^) + T ^Ah^)(x) =■■■. 

Thus when A — ► +oo, it seems that the main contribution to (e A h) comes from —h'. If wc replace 
(cA^k) m ( 8.10| )-( 8.12 ) above by — ^ and drop the term e A (x — y), (p^Sj ) is equal to (8.1) of if 



the notations Z^jv and SnD there are exchanged. This seems to be an indication that the kernel 
Kn(x, y; A) actually converges to the kernel (8.1) of p9| , the /3 = 4 symplectic ensemble, as A — > +oo. 



Nevertheless, Proposition 1.1 shows that the determinant det(l + ifjj 4 ' ) /) converges to the corresponding 



determinant for /3 = 4 symplectic ensemble as A — > oo for a proper class of functions /. 

We finish this section with some properties of the operator e A which can be checked easily. Let h 
be a function in the Schawrz class. 

• Let g = e A h. Then g" - A 2 g = 2h'. 

• If h, in addition to the smooth and decay conditions, satisfies J m h(x)ds = 0, we have 

i^ A h){x) = h'(x) - ^ J* h{t)dt. (8.14) 

References 

[1] M. Abramowitz and I. Stegun. Handbook of Mathematical Functions. Dover Publications, New 
York, 1965. 

[2] M. Adler, P. Forrester, T. Nagao, and P. van Moerbeke. Classical skew orthogonal polynomials 
and random matrices. J. Statist. Phys., 99(1-2):141-170, 2000. 



39 



[3] J. Baik. Limiting distributions for one dimensional totally asymmetric simple exclusion process, 
in preparation. 

[4] J. Baik, P. Dcift, and K. Johansson. On the distribution of the length of the longest increasing 
subsequence of random permutations. J. Amer. Math. Soc, 12(4):1119— 1178, 1999. 

[5] J. Baik and E. M. Rains. Algebraic aspects of increasing subsequences. Duke Math. J., 109(l):l-65, 
2001. 

[6] J. Baik and E. M. Rains. The asymptotics of monotone subsequences of involutions. Duke Math. 
J., 109(2):205-281, 2001. 

[7] J. Baik and E. M. Rains. Symmetrized random permutations. In P. Bleher and A. Its, editors, 
Random Matrix Models and Their Applications, volume 40 of Mathematical Sciences Research 
Institute Publications, pages 1-19. Cambridge, 2001. 

[8] N. G. de Bruijn. On some multiple integrals involving determinants. J. Indian Math. Soc. (N.S.), 
19:133-151, 1955. 

[9] P. Deift, T. Kriecherbauer, K. McLaughlin, S. Venakides, and X. Zhou. Strong asymptotics of or- 
thogonal polynomials with respect to exponential weights. Comm. Pure Appl. Math., 52(12):1491- 
1552, 1999. 

[10] P. Deift, T. Kriecherbauer, K. McLaughlin, S. Venakides, and X. Zhou. Uniform asymptotics for 
polynomials orthogonal with respect to varying exponential weights and applications to universality 
questions in random matrix theory. Comm. Pure Appl. Math., 52(11):1335-1425, 1999. 

[11] P. Deift and X. Zhou. A steepest descent method for oscillatory Riemman-Hilbcrt problems; 
asymptotics for the MKdV equation. Ann. of Math., 137:295-368, 1993. 

[12] A. Fokas, A. Its, and V. Kitaev. Discrete Painleve equations and their appearance in quantum 
gravity. Comm. Math. Phys., 142:313-344, 1991. 

[13] P. Forrester. Painleve transcendent evaluation of the scaled distribution of the smallest eigenvalue 
in the Laguerre orthogonal and symplectic ensembles, nlin. SI/0005064 ; http://xxx.lanl.gov/abs/. 

[14] P. Forrester and E. Rains. Inter-relationships between orthogonal, unitary and symplectic ma- 
trix ensembles. In P. Bleher and A. Its, editors, Random Matrix Models and Their Applications, 
volume 40 of Mathematical Sciences Research Institute Publications, pages 171-207. Cambridge, 
2001. 

[15] P. Forrester and E. M. Rains. Correlations for superimpositions and decimations of laguerre and 
jacobi orthogonal matrix ensembles with a parameter, in preparation. 

[16] P. Forrester and N. Witte. Application of the r-function theory of Painleve equations to random ma- 
trices: PVI, the JUE, CyUE, cJUE and scaled limits, math-ph/0204008 ; http://xxx.lanl.gov/abs/. 



40 



[17] P. Forrester and N. Witte. r-Function evaluation of gap probabilities in orthogonal and symplectic 
matrix ensembles, math-ph/0203049 ; http://xxx.lanl.gov/abs/. 

[18] K. Johansson. On random matrices from the compact classical groups. Ann. of Math., 145(3):519- 
545, 1997. 

[19] K. Johansson. Shape fluctuations and random matrices. Comm. Math. Phys., 209(2):437-476, 
2000. 

[20] I. Johnstone. On the distribution of the largest eigenvalue in principal components analysis. Ann. 
Statist, 29(2):295-327, 2001. 

[21] T. Liggett. Stochastic interacting systems : contact, voter and exclusion processes, volume 324 of 
Grundlehren der mathematischen Wissenschaften. Springer, New York, 1999. 

[22] M. Mchta. Random matrices. Academic press, San Diago, second edition, 1991. 

[23] E. M. Rains. Correlation functions for symmetrized increasing subsequences, math. CO/0006097 ; 
http://xxx.lanl.gov/abs/. 

[24] H. Spohn and M. Prahofcr. Statistical self-similarity of one-dimensional growth processes. Phys. 
Rev. Lett, 84:4882-4885, 2000. 

[25] R. P. Stanley. Enumerative Combinatorics, volume 2. Cambridge University Press, Cambridge, 
United Kingdom, 1999. 

[26] C. Tracy and H. Widom. Frcdholm determinants, differential equations and matrix models. Comm. 
Math. Phys., 163:33-72, 1994. 

[27] C. Tracy and H. Widom. Level-spacing distributions and the Airy kernel. Comm. Math. Phys., 
159:151-174, 1994. 

[28] C. Tracy and H. Widom. On orthogonal and symplectic matrix ensembles. Comm. Math. Phys., 
177:727-754, 1996. 

[29] C. Tracy and H. Widom. Correlation functions, cluster functions and spacing distributions for 
random matrices. J. Statist. Phys., 92:809-835, 1998. 

[30] P. van Moerbeke. Integrable lattices: random matrices and random permutations. In P. Blchcr 
and A. Its, editors, Random Matrix Models and Their Applications, volume 40 of Mathematical 
Sciences Research Institute Publications, pages 321-406. Cambridge, 2001. 

[31] H. Widom. On the relation between orthogonal, symplectic and unitary matrix ensembles. J. 
Statist. Phys., 94(3-4):347-363, 1999. 



41 



